Histopathological image analysis : a review

Over the past decade, dramatic increases in computational power and improvement in image analysis algorithms have allowed the development of powerful computer-assisted analytical approaches to radiological data. With the recent advent of whole slide digital scanners, tissue histopathology slides can now be digitized and stored in digital image form. Consequently, digitized tissue histopathology has now become amenable to the application of computerized image analysis and machine learning techniques. Analogous to the role of computer-assisted diagnosis (CAD) algorithms in medical imaging to complement the opinion of a radiologist, CAD algorithms have begun to be developed for disease detection, diagnosis, and prognosis prediction to complement the opinion of the pathologist. In this paper, we review the recent state of the art CAD technology for digitized histopathology. This paper also briefly describes the development and application of novel image analysis technology for a few specific histopathology related problems being pursued in the United States and Europe

Improving Prostate Cancer Detection with Breast Histopathology Images

Deep neural networks have introduced significant advancements in the field of machine learning-based analysis of digital pathology images including prostate tissue images. With the help of transfer learning, classification and segmentation performance of neural network models have been further increased. However, due to the absence of large, extensively annotated, publicly available prostate histopathology datasets, several previous studies employ datasets from well-studied computer vision tasks such as ImageNet dataset. In this work, we propose a transfer learning scheme from breast histopathology images to improve prostate cancer detection performance. We validate our approach on annotated prostate whole slide images by using a publicly available breast histopathology dataset as pre-training. We show that the proposed cross-cancer approach outperforms transfer learning from ImageNet dataset.Comment: 9 pages, 2 figure

Delta radiomic patterns on serial bi-parametric MRI are associated with pathologic upgrading in prostate cancer patients on active surveillance: preliminary findings

ObjectiveThe aim of this study was to quantify radiomic changes in prostate cancer (PCa) progression on serial MRI among patients on active surveillance (AS) and evaluate their association with pathologic progression on biopsy.MethodsThis retrospective study comprised N = 121 biopsy-proven PCa patients on AS at a single institution, of whom N = 50 at baseline conformed to the inclusion criteria. ISUP Gleason Grade Groups (GGG) were obtained from 12-core TRUS-guided systematic biopsies at baseline and follow-up. A biopsy upgrade (AS+) was defined as an increase in GGG (or in number of positive cores) and no upgrade (AS−) was defined when GGG remained the same during a median period of 18 months. Of N = 50 patients at baseline, N = 30 had MRI scans available at follow-up (median interval = 18 months) and were included for delta radiomic analysis. A total of 252 radiomic features were extracted from the PCa region of interest identified by board-certified radiologists on 3T bi-parametric MRI [T2-weighted (T2W) and apparent diffusion coefficient (ADC)]. Delta radiomic features were computed as the difference of radiomic feature between baseline and follow-up scans. The association of AS+ with age, prostate-specific antigen (PSA), Prostate Imaging Reporting and Data System (PIRADS v2.1) score, and tumor size was evaluated at baseline and follow-up. Various prediction models were built using random forest (RF) classifier within a threefold cross-validation framework leveraging baseline radiomics (Cbr), baseline radiomics + baseline clinical (Cbrbcl), delta radiomics (CΔr), delta radiomics + baseline clinical (CΔrbcl), and delta radiomics + delta clinical (CΔrΔcl).ResultsAn AUC of 0.64 ± 0.09 was obtained for Cbr, which increased to 0.70 ± 0.18 with the integration of clinical variables (Cbrbcl). CΔr yielded an AUC of 0.74 ± 0.15. Integrating delta radiomics with baseline clinical variables yielded an AUC of 0.77 ± 0.23. CΔrΔclresulted in the best AUC of 0.84 ± 0.20 (p < 0.05) among all combinations.ConclusionOur preliminary findings suggest that delta radiomics were more strongly associated with upgrade events compared to PIRADS and other clinical variables. Delta radiomics on serial MRI in combination with changes in clinical variables (PSA and tumor volume) between baseline and follow-up showed the strongest association with biopsy upgrade in PCa patients on AS. Further independent multi-site validation of these preliminary findings is warranted

Image segmentation with implicit color standardization using spatially constrained expectation maximization: Detection of nuclei

Abstract. Color nonstandardness -the propensity for similar objects to exhibit different color properties across images -poses a significant problem in the computerized analysis of histopathology. Though many papers propose means for improving color constancy, the vast majority assume image formation via reflective light instead of light transmission as in microscopy, and thus are inappropriate for histological analysis. Previously, we presented a novel Bayesian color segmentation algorithm for histological images that is highly robust to color nonstandardness; this algorithm employed the expectation maximization (EM) algorithm to dynamically estimate -for each individual image -the probability density functions that describe the colors of salient objects. However, our approach, like most EM-based algorithms, ignored important spatial constraints, such as those modeled by Markov random field (MRFs). Addressing this deficiency, we now present spatially-constrained EM (SCEM), a novel approach for incorporating Markov priors into the EM framework. With respect to our segmentation system, we replace EM with SCEM and then assess its improved ability to segment nuclei in H&E stained histopathology. Segmentation performance is evaluated over seven (nearly) identical sections of gastrointestinal tissue stained using different protocols (simulating severe color nonstandardness). Over this dataset, our system identifies nuclear regions with an area under the receiver operator characteristic curve (AUC) of 0.838. If we disregard spatial constraints, the AUC drops to 0.748

Hierarchical Normalized Cuts: Unsupervised Segmentation of Vascular Biomarkers from Ovarian Cancer Tissue Microarrays

Research has shown that turner vascular markers (TVMs) may serve as potential OCa biomarkers for prognosis prediction. One such TVM is ESM-1, which can be visualized by staining ovarian Tissue Microarrays (TMA) with in antibody to ESM-1. The ability to quickly and quantitatively estimate vascular stained regions may yield an image based metric linked to disease survival and outcome. Automated segmentation of the vascular stained regions on the TMAs. however, is hindered by the presence of spuriously stained false positive regions. In this paper, we present a general, robust and efficient unsupervised segmentation algorithm, termed Hierarchical Normalized Cuts (HNCut), and show its application in precisely quantifying the presence and extent of a TVM on OCa TMAs. The strength of HNCut is in the use of a hierarchically represented data structure that bridges the mean shift (MS) and the normalized cuts (NCut,) algorithms. This allows HNCut to efficiently traverse a pyramid of the input image at various color resolutions, efficiently and accurately segmenting the object class of interest (in this case ESM-1. vascular stained regions) by simply annotating half a, dozen pixels belonging to the target; class. Quantitative and qualitative analysis of our results, using 100 pathologist annotated samples across multiple studies, prove the superiority of our method (sensitivity 81%, Positive predictive value (PPV), 80%) versus a popular supervised learning technique, Probabilistic Boosting Trees (sensitivity, PPV of 76% and 66%)

Deep learning-based prediction of response to HER2-targeted neoadjuvant chemotherapy from pre-treatment dynamic breast MRI: A multi-institutional validation study

Predicting response to neoadjuvant therapy is a vexing challenge in breast cancer. In this study, we evaluate the ability of deep learning to predict response to HER2-targeted neo-adjuvant chemotherapy (NAC) from pre-treatment dynamic contrast-enhanced (DCE) MRI acquired prior to treatment. In a retrospective study encompassing DCE-MRI data from a total of 157 HER2+ breast cancer patients from 5 institutions, we developed and validated a deep learning approach for predicting pathological complete response (pCR) to HER2-targeted NAC prior to treatment. 100 patients who received HER2-targeted neoadjuvant chemotherapy at a single institution were used to train (n=85) and tune (n=15) a convolutional neural network (CNN) to predict pCR. A multi-input CNN leveraging both pre-contrast and late post-contrast DCE-MRI acquisitions was identified to achieve optimal response prediction within the validation set (AUC=0.93). This model was then tested on two independent testing cohorts with pre-treatment DCE-MRI data. It achieved strong performance in a 28 patient testing set from a second institution (AUC=0.85, 95% CI 0.67-1.0, p=.0008) and a 29 patient multicenter trial including data from 3 additional institutions (AUC=0.77, 95% CI 0.58-0.97, p=0.006). Deep learning-based response prediction model was found to exceed a multivariable model incorporating predictive clinical variables (AUC < .65 in testing cohorts) and a model of semi-quantitative DCE-MRI pharmacokinetic measurements (AUC < .60 in testing cohorts). The results presented in this work across multiple sites suggest that with further validation deep learning could provide an effective and reliable tool to guide targeted therapy in breast cancer, thus reducing overtreatment among HER2+ patients.Comment: Braman and El Adoui contributed equally to this work. 33 pages, 3 figures in main tex

Breast Cancer Immunohistochemical Image Generation: a Benchmark Dataset and Challenge Review

For invasive breast cancer, immunohistochemical (IHC) techniques are often used to detect the expression level of human epidermal growth factor receptor-2 (HER2) in breast tissue to formulate a precise treatment plan. From the perspective of saving manpower, material and time costs, directly generating IHC-stained images from hematoxylin and eosin (H&E) stained images is a valuable research direction. Therefore, we held the breast cancer immunohistochemical image generation challenge, aiming to explore novel ideas of deep learning technology in pathological image generation and promote research in this field. The challenge provided registered H&E and IHC-stained image pairs, and participants were required to use these images to train a model that can directly generate IHC-stained images from corresponding H&E-stained images. We selected and reviewed the five highest-ranking methods based on their PSNR and SSIM metrics, while also providing overviews of the corresponding pipelines and implementations. In this paper, we further analyze the current limitations in the field of breast cancer immunohistochemical image generation and forecast the future development of this field. We hope that the released dataset and the challenge will inspire more scholars to jointly study higher-quality IHC-stained image generation.Comment: 13 pages, 11 figures, 2table